Accepted Jan 5, 2021; Epub ahead of print Jan 20, 2021

J Rehabil Med 2021; 53: jrm00148

Correspondence address: Margit Alt Murphy, Institute of Neuroscience and Physiology, Clinical Neuroscience, Rehabilitation Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden.
E-mail: margit.alt-murphy@neuro.gu.se

DOI: 10.2340/16501977-2790

Currently available clinical practice guidelines do not specify which outcome measures should be used at which time-points for people after stroke. This study describes the implementation process and evaluation of a clinical practice guideline developed for the assessment of upper extremity function after stroke. The guideline is based on recent research evidence and defines the assessments, and the time-points at which the assessments should be performed. An evaluation survey showed that clinicians valued the clear structure of the guideline and found it useful for prognosis and treatment planning. Robust evidence, and active involvement of clinicians and leaders, were important elements of implementation. The guideline will potentially improve the quality of rehabilitation through increased knowledge of prognosis and treatment effects, based on the assessment of arm function in people with stroke, thereby enabling a more evidence-based, consistent, and individually tailored rehabilitation.

Considerable efforts have been made recently in the field of stroke rehabilitation to develop evidence-based agreed guidelines for upper extremity assessment (1–4). Advances in predicting stroke outcomes, in terms of motor function and activity, have been the primary driver of this development (5–7). There is, however, a clear evidence–practice gap, since the use of recommended upper extremity outcome measures in day-to-day clinical practice is sparse. It is well recognized that valid, reliable and responsive outcome measures, performed at pre-defined time-points after stroke, are required for effective rehabilitation (2–4). Currently available clinical guidelines recommend the use of standardized outcome measures, but often do not specify what outcome measures should be used, at what frequency, or in what settings (2).

The implementation of standardized recommended upper extremity outcome measures in clinical routine practice takes time and effort. Increased and consistent use of such measures is, however, required to enable person-centred informed clinical decision-making throughout the rehabilitation pathway, and thereby improve patient outcomes. Overall adherence to stroke guidelines varies, but, in general, it is greater when the implementation process includes systematic and well-defined activities (8–10). There are no recognized “gold standard” implementation activities, although multifaceted interventions involving educational outreach and a structured theoretical approach have been suggested to work best (8, 9). Organizational and multidisciplinary team factors, staff beliefs regarding the guidelines, integration of patient-centred recommendations into practice, awareness of guidelines, changing routines, and necessary time investment, are known factors affecting adherence (8, 10).

The successful implementation of evidence into practice is dependent on the quality of evidence, the context, and how the evidence is introduced into practice (facilitators) (11). These 3 key elements, being part of the Promoting Action on Research Implementation in Health Services (PARIHS) theoretical framework, have been employed widely in different implementation activities (11, 12). This theoretical model prerequisites that the evidence is robust, practitioners agree with it, and the context is receptive, including the formal leaders, and that appropriate facilitation is ensured (11, 13). The Knowledge to Action (KTA) framework is another theoretical tool that has been widely used to make the process of knowledge translation into practice more systematic (14). The KTA emphasizes the importance of adapting knowledge to the local context, of involving stakeholders, and of being aware of barriers, facilitators and user needs (14).

The literature is extensive regarding the implementation of stroke guidelines into clinical practice (8, 9), but only a few studies have specifically targeted assessment and use of standardized outcome measures (15, 16). More recent work on recommendations regarding upper extremity outcome measures (3) also imply a need to move this research evidence into stroke rehabilitation practice.

The aim of this study was to describe and evaluate the implementation process and adherence to an evidence-based clinical practice guideline (CPG) for physical therapists (PT) and occupational therapists (OT) in the assessment of upper extremity function and activity during acute and subacute stroke rehabilitation.

Setting and context

The implementation process was initiated in 2014 as part of the strategic work of Research & Development (R&D) at the Department of Occupational Therapy and Physiotherapy, Sahlgrenska University Hospital, Gothenburg, Sweden. Approximately 317 PT and OT work at the hospital, of whom approximately 23 PT and 22 OT work in stroke rehabilitation at 4 different hospital sites. Three hospital sites (Sahlgrenska, Östra, Mölndal) provide acute inpatient rehabilitation at Stroke Units, and one (Högsbo) provides specialized neurological rehabilitation after the acute phase (inpatient and outpatient rehabilitation).

Four clinical researchers with doctoral degrees (2 PT and 2 OT) were strategically selected for the assignment and comprised the R&D Stroke group. The group members had extensive clinical experience of stroke rehabilitation within the organization (15–45 years) and a high level of research expertise covering 4 main aspects of stroke rehabilitation (upper extremity, postural control/walking, cognition, and activities of daily living (ADL)). The group aimed to address evidence-practice gaps by systematically mapping the physiotherapy and occupational therapy clinical practice, summarizing the evidence, and developing and implementing locally adapted clinical guidelines for stroke rehabilitation. The R&D Stroke group produced semi-annual and annual reports, which were disseminated to the management group and clinicians. Support from the management group, along with collaborative joint activities between the members of the group, were an integrated part of the implementation process. This paper only presents the work concerning the development and implementation of evidence-based CPG for upper extremity assessment, led by one of the authors (MAM).

The implementation process was supported by well-established theoretical frameworks, the PARIHS (11, 12) and the KTA (14), as well as the International Classification of Functioning, Disability and Health (ICF) (17). The KTA process is complex and dynamic, but can be divided into 2 main concepts: knowledge creation and action (14). In the current study, the implementation process started with knowledge creation, which included mapping the assessments used in clinical practice, followed by revision of the evidence-base on upper extremity assessments (Fig. 1). This knowledge was subsequently evaluated and adapted to the local context through a variety of actions. The key elements of the PARISH model (strength of the evidence, context, and facilitation) were used to guide the process in terms of evaluation of barriers to knowledge use and in selection of tailored interventions. Early use of the guideline was monitored through various actions and, finally, evaluated using a survey. Long-term sustainable use of the developed guideline is still an ongoing process. The Appraisal of Guidelines for Research & Evaluation II (AGREE) reporting checklist was used to provide methodological support for guideline development (18).

Mapping clinical practice

A survey was sent to all clinicians working with stroke rehabilitation. The survey was answered collectively by each PT and OT unit, and followed up by open group discussions moderated by a clinical researcher from the R&D Stroke group. The questions of the survey included aspects on how the assessments were done; which standardized assessments were used, which profession was responsible for which assessment, and how the inter-disciplinary co-operation was organized between OT and PT. Results from discussions were summarized using the strengths, opportunities, weaknesses and threats (SWOT) analysis tool (19).

Evidence-base on upper extremity assessments

The literature search was performed by the clinical researcher (MAM) in PubMed and complemented by a search in Google Scholar database for CPG in November 2015. The aim of the search was to identify outcome measures with strong psychometric properties and clinical utility recommended for stroke rehabilitation. The search was limited to peer-reviewed systematic reviews, CPG, and recommendations, published in English between 2005 and 2015, with the following search terms: stroke, upper extremity, arm, upper limb, assessment, outcome measure, instrument, scale, test, national guideline, practice guideline and recommendations. Only measures available in Swedish were considered. The results were summarized and disseminated to clinicians in a written report and through a series of meetings.

Development of upper extremity clinical practice guideline for stroke

At this stage, a series of information meetings was organized to provide a preliminary plan for implementation work and to encourage interested clinicians to become involved. Subsequently, each OT and PT group nominated at least one clinician to be included in the upper extremity clinical reference group. The final selection was coordinated with the other ongoing activities, and considered aspects such as professional role characteristics, facilitation skills, group dynamics and workload. The role of the reference group member was formalized through the organization. The clinical reference group, consisting of 4 PT and 4 OT, had approximately 6 yearly meetings led by the clinical researcher (MAM). The group provided valuable input and critical feedback on the first and revised drafts of the guidelines, contributed to the action plan, mediated and facilitated information flow at their respective unit, and facilitated local meetings and workshops. Consensus-based decision-making was used at the reference group meetings to agree on the final content of the guidelines. A series of meetings was also organized at each unit to allow open discussions and direct input and feedback on the content of guidelines from all clinicians. Practical workshops on specific standardized outcome measures were organized at all sites. Supporting materials were produced for new outcome measures, e.g. an educational video on how to perform the Fugl-Meyer Assessment of Upper Extremity (FMA-UE) (www.neurophys.gu.se/rehabmed).

Implementation

Implementation of the upper extremity assessment guideline was performed gradually, depending on the working situation at each unit. The final guideline for upper extremity was available to clinicians in January 2019 and published online (http://hittadokument.vgregion.se) in September 2019 (a summary in English is available in Supplement 1). Subsequently, patient case discussions were conducted within each unit, based on real documented unidentified patient data from the medical charts provided by the clinicians to support clinicians in the use and interpretation of the guideline.

Evaluation

A survey, developed by the clinical researcher (MAM), aimed to gather and summarize clinicians’ experiences and perceptions on the implementation process and to evaluate adherence to the guideline. After piloting in a separate group (2 PT and 1 OT with extensive clinical and research experience), and approval from the management group, the survey was delivered and completed individually and anonymously by clinicians working with stroke in March 2020. A 4-grade Likert scale (good, rather good, rather bad, bad or agree fully, agree partly, agree poorly, don’t agree) and predefined intervals were used in ratings, and reported descriptively as percentages. The free-text comments were organized and analysed using the SWOT analysing tool (19).

Mapping clinical practice

A mix of standardized (Table I) and non-standardized assessments of upper extremity function and activity were used in clinical practice in 2014. The SWOT analysis and group discussions revealed that high level of clinical expertise, collective support, and good collaboration between PT and OT strengthened the clinical practice. Even when therapists were, in general, satisfied with available clinical stroke guidelines, they wanted to have more detailed guidance on selection of assessments and interventions. Assessment was considered to be important, particularly when it could be integrated into daily treatment sessions. Not having enough time to perform standardized assessments, particularly in the acute stroke unit, was perceived as a threat. Furthermore, the assessment of upper extremity function was not prioritized when time was limited. Unclear division between the OT and PT roles, particularly in upper extremity rehabilitation, was seen as a weakness. For example, some assessments, such as assessment of sensory function, were performed by both professions, while assessments and active training of motor function could receive a low priority if the OT prioritized interventions targeting ADL and cognition and the PT targeted mobility and walking ability. The rehabilitation recommendations based on the PT and OT assessments were not always considered in discharge and continuous rehabilitation planning, which was perceived as frustrating. Communication within the rehabilitation team was challenging, and therefore considered as a threat.

Evidence base on upper extremity assessments

The literature search identified an overview of systematic reviews on upper extremity outcome measures in stroke (1). This paper covered the topic for years up to 2014, and included 13 systematic reviews. An additional search covering the years 2014 and 2015 resulted in 45 articles, of which 7 were included, but only 3 provided a recommendation on upper extremity outcome measures; and all of those were non-systematic reviews (20–22). Through the complementary search in Google, among the publicly available selected national clinical guidelines (Swedish, Dutch, UK, Australia, South Africa, Singapore, New Zeeland, Estonia), the Dutch clinical guideline for physical therapy in stroke was the only national guideline identified that recommended specific upper extremity outcome measures for clinical practice (23). A consensus-based recommendation, which provided specific recommendations on upper extremity outcome measures, was also included (24). All outcome measures (used in 2014 and those identified through literature search) along with the extracted information from the literature search and the decision on inclusion in the upper extremity assessment guideline, as primary or add-on measures, are shown in Table I.

Since the time available for assessments in the acute stage of stroke is often limited, 2 additional prognostic short screening tests (Shoulder Abduction, Finger Extension (SAFE), indicating shoulder abduction and finger extension (25), and ARAT-2 (26)), were considered as potential scales for use in the acute stroke unit. SAFE assesses the muscle strength of shoulder abduction and finger extension using the clinically well-established Medical Research Council 0–5 manual muscle testing scale (25), and ARAT-2 is a sum-score of 2 items of the Action Research Arm test (ARAT) (26, 27).

Development of upper extremity assessment guideline for stroke

During the guideline development, consensus-based core recommendations for standardized measurements of motor recovery in stroke trials were published (3), along with a systematic review of clinical guidelines (2) and a Delphi consensus study on upper extremity evaluation in stroke (28). These recommendations were in line with our selection of outcome measures to be included in the guidelines. Initially, 6 measures, covering the body function and activity domain of the ICF, were considered as primary assessments for stroke: SAFE, ARAT-2 (2-items of the Action Research Arm Test), Fugl-Meyer Assessment of Upper Extremity (FMA-UE), ARAT, Box and Block Test (BBT), 9-Hole Peg Test (9HPT), and grip strength (Jamar hand dynamometer). SAFE and ARAT-2 are short screening tests that can be used for early prediction and as guidance for the selection of further assessments. The FMA-UE provides comprehensive information regarding motor impairment, and the ARAT provides information on activity capacity limitation. The BBT is a simple test of gross motor dexterity, and the 9HPT is a test of fine motor dexterity.

The BBT, 9HPT and grip strength were already well established in clinical practice. The new measures, SAFE and ARAT-2, were considered feasible for use in acute settings. Clinicians were interested in the FMA-UE and ARAT, but only a few clinicians had previous experience on these tests. PT found that FMA-UE could add valuable information on upper extremity motor function, since the available assessments used at this stage were limited. However, to perform both FMA-UE and ARAT in clinical settings was not considered feasible. OT already used some of the recommended standardized measures, and prioritized observational assessments in everyday activities, such as drinking from a glass, and manipulation of real everyday objects. The majority of OT found the ARAT to be more suitable for PT, since it does not provide information on how the patient is functioning in real-life activities. Thus, the final guideline included SAFE, ARAT-2, FMA-UE, BBT, 9HPT and grip strength as primary assessments, with ARAT as a complementary test.

The research evidence was clear that, in order to follow the recovery process and provide an informed prediction, the agreed time-points for assessments need to be relative to stroke onset (3). The first standardized assessment of motor function was recommended to be performed within 7 days of stroke onset (3). To achieve full use of the predictive screening tests, such as, SAFE and ARAT-2, the assessments should be performed within 3 days post-stroke (7, 25–27). There is also a clear recommendation to repeat the assessment at discharge (2). Since the prediction of upper extremity function in patients with initial poor motor function is limited when only clinical assessments are available, an additional follow-up assessment at approximately 4 weeks after stroke onset would provide some more information (3). Commonly, 3-, 6- and 12-month follow-up assessments are also recommended post-stroke (2, 3). A schematic flowchart on the assessment process is shown in Fig. 2.

Based on the recommendations and evidence (7, 25), an adapted prediction algorithm was developed for clinical use (Figs 3 and 4). The Predict Recovery Potential (PREP2) algorithm was used for patients with relatively good initial SAFE score (≥ 5 points) (7). For patients with poor initial SAFE score (< 5 points), the algorithm was adapted. Here, an additional assessment of FMA-UE was added at 4 weeks post-stroke to guide the prediction. At 4 weeks, a cut-off of FMA ≥ 20 points was used to indicate that some recovery of upper extremity function could be achieved at 6 months post-stroke (29). It is important to note that, for those who remained below this cut-off, the prognosis was defined as uncertain and not poor. A similar algorithm, based on previous research, was developed for the ARAT-2 scores (26, 27) (Fig. 4).

Evaluation

The survey was answered by 44 clinicians (21 OT and 23 PT, response rate 98%, mean 12.5 years of working experience) (Supplement 2). A majority of respondents (62%) had been working in stroke rehabilitation during the entire implementation process, and only 2 had less than one year in stroke rehabilitation. A majority of the respondents (80–96%) found that information provided during the implementation was good or rather good. Almost all were satisfied with the implementation process and the clinical reference group representatives. Approximately 75% of respondents found that they were able to find time to take part in the information provided and learn the new instruments. Approximately 96% of respondents indicated that learning SAFE was easy or rather easy, whereas 90% and 83% reported the same regarding ARAT-2 and FMA-UE, respectively. Approximately 91% indicated that it worked well to use the SAFE in clinical routine, whereas only 62% and 65% stated that about the ARAT-2 and FMA-UE, respectively. Approximately 52% of respondents stated that adherence to the defined assessment time-points as well as interpretation of the test results to guide treatment planning, was good or rather good, while 43% stated that it was rather bad.

More than 60% of respondents agreed that the stroke guideline had increased their focus and knowledge-base regarding upper extremity assessment, and 75% indicated that the guideline provided good structure and a useful tool for prediction. Interestingly, almost 90% of respondents reported that the time spent on assessment did not influence the time they spent on treatment.

Most of the clinicians had used the new measures between 6 and 20 times. Most commonly, approximately 6–10 patient assessments were needed for the clinician to feel comfortable with the FMA-UE, whereas only 3–5 patient assessments were needed for SAFE and ARAT-2. Assessment with the FMA-UE was reported to take approximately 11–20 min, whereas the ARAT-2 took approximately 6–10 min, and SAFE 1–5 min.

A total of 27 clinicians (61%) left at least one free-text comment. In total, 69 comments were made (23 positive, 25 negative, and 21 neutral). The summarized comments, organized in a SWOT plot, are shown in Table II. The implementation process, including information, materials, feedback and performed activities, was valued by clinicians, even when the process was long. The guideline provided structured and clear information to support prognosis and treatment planning. The short screening tests were easier to implement than the comprehensive tests. To perform assessments at pre-defined time-points after stroke onset was new to many clinicians and required a change in routines. The implementation of ARAT-2 needs, however, additional activities to improve the adherence. High level of staff work rotation, and the need to prioritize other assessments during the first week after stroke, hindered adherence. Continuous training was considered important for long-term adherence.

This study provides a comprehensive description of the development process and outcomes in implementation of an evidence-based CPG for OT and PT in the assessment of upper extremity function and activity during the acute and subacute stage of stroke rehabilitation. This was a collective work, initiated by the Department of Occupational therapy and Physiotherapy at Sahlgrenska University Hospital, with the aim of bridging the evidence–practice gap and providing a tool for clinicians that would guide and support clinical decision-making. The final guideline included a structured assessment plan with defined time-points and a set of standardized outcome measures covering the main aspects of upper extremity function and activity. The upper extremity guideline was integrated for both professions, although the roles were distinguished and adapted to local practice needs in order to ensure effectiveness in assessment and avoid repeating work. The main focus was on covering all important aspects of patient assessment, and not on which profession is performing which assessment.

The evidence base for upper extremity assessment in stroke was relatively strong. The evidence from systematic reviews, consensus-based international expert recommendations and published CPG was in agreement, showing that early and repeated standardized assessments relative to stroke onset are required in order to follow the recovery and to provide an optimal informed prognosis of motor outcome (1–3, 7, 25). It was also clear that, when only clinical assessments are available, repeated assessments at defined time-points can be used to improve the precision of prognosis, particularly for patients with poor motor function early after stroke onset (7, 29).

Regarding the specific outcome measures, the recommendation of the FMA-UE to assess motor function in stroke is strong (1, 3, 4). FMA-UE was not well known to clinicians, although many had heard of it, and a few PT had used a modified version of it decades ago. Here, mapping of the practice was important, since it clearly showed that, since it clearly showed that, at the time of the current work, PT did not have any specific tool to assess upper extremity motor function. Even though the FMA-UE scale was initially considered cumbersome, old-fashioned, and not very functional, PT showed interest and were willing to try it out. The existing research provided also information on how the scores can be interpreted and used in clinical decision-making. This information was critical for implementation of the FMA-UE in clinical settings. Finally, acceptance of the FMA-UE for inclusion as a primary assessment in the stroke guidelines was satisfactory, although it was recognized that the scale required extra time and effort to learn.

Based on the available evidence, the ARAT would have been the first choice for assessment of upper extremity activity capacity. Despite good evidence, the implementation process showed that it was difficult to include this scale in the guidelines as a primary assessment at this stage of implementation. The PT found that it was not feasible to include 2 comprehensive new instruments at the same time, and OT prioritized other assessments of ADL. A further argument against the use of the ARAT was the need to acquire multiple test kits, and that it might be difficult to find suitable space and time in the acute settings to perform the test.

The short screening instrument, SAFE, was rapidly accepted and implemented, mainly because the scoring was known to PT and interpretation was simple. The ARAT-2 was new to the OT, and the item “pour water from a glass” required that the patient was sitting at a table. Implementation of the ARAT-2 took longer, and more support is needed, as seen from the results of the final survey.

In agreement with established theoretical frameworks of implementation and knowledge translation research, this study found that solid evidence-base, facilitation and receptive context were the 3 crucial elements of successful implementation (11, 14, 30). Notably, assessments with a strong and robust evidence-base were more readily accepted by clinicians. An effort was made to tailor and adapt information from research to local clinical practice in order to make it more available to clinicians (9, 15). It was found that information needed to be presented gradually and repeated over time in different forms. This approach allowed most of the clinicians to take part in the implementation work. The discussions and critique that emerged from this sharing was the driving force of the knowledge translation process.

Clinicians also appreciated the tailored materials and workshops summarizing the research evidence, as provided by the R&D clinical researcher (MAM). Tailored resources, broken down into manageable sections, are often valued by the therapists, as this saves them time to undertake other tasks (15, 31). In a busy clinical setting, putting therapists in the position of running the implementation tasks, in addition to their daily work is not usually achievable (31). Thus, the expertise of the clinical researcher, in combination with previous clinical experience, were important perquisites for successful implementation within the boundaries of organization. The involvement of clinical representatives acting as a link between the clinical researcher and clinicians was also a necessity in the implementation process. The use of facilitators and tailoring of interventions for local settings have been shown to be the most essential aspects of a successful implementation strategy (9).

Finally, the role of context should not be underestimated. The implementation work had relevance to the organization, and was initiated by the formal leaders, which provided stability and trustworthiness. However, the clinical researcher, being an expert on upper extremity stroke rehabilitation, had the operational role of planning, coordinating and executing the implementation activities independently. This set-up increased commitment and allowed successful implementation despite the limited resources available at the hospital. In general, the clinical expertise among therapists, having a large joint organization for PT and OT, as well as the therapists’ readiness to work in an evidence-based way were also contextual factors facilitating implementation.

Limitations and future challenges

This study showed that the implementation process is complex and takes time. This can be a limitation itself when the knowledge-base is constantly changing and needs to be updated almost in parallel with guideline development. Furthermore, time was also an important factor in making a change of practice possible. Clinicians had to be involved, informed and have the possibility to reflect and test the suggested changes. Such change takes time, particularly in a large organization with limited resources. Another limitation was that patients’ views and preferences were not included in the implementation process.

The implementation process is ongoing. The feedback from the clinicians revealed a few areas that need extra attention in future work. A plan for continuous education, involving regular updates and introduction to new colleagues, is needed. The content of the guideline needs also to be updated and revised on a regular basis e.g. every second year. There is a need to establish local clinical facilitators in order to retain the guideline as a natural part of daily clinical practice. In the long term, continuous recruitment of new dedicated clinical researchers is needed. Finally, continuous support is also needed from the organization advocating the use of evidence-based interventions. Only then will we have well-informed therapists who can make a difference in the clinical decision-making process together with the patient. The long-term feasibility and impact on patient outcomes, however, need to be evaluated in the coming years. Finally, it must also be recognized that the guideline should primarily be used as guidance to support clinical decision-making, and that it will not include all possible aspects of assessment of the upper extremity in people with stroke.

Conclusion

This study showed that the robustness of the evidence-base, dedicated involvement of facilitators, and receptive context, including support from formal leaders, were all necessary for successful implementation of the guideline. To allow sufficient time for the process was important, in order to achieve informed consensus and acceptance of the content of the guideline. The final CPG for PT and OT for the assessment of upper extremity function and activity during acute and subacute stroke rehabilitation was well-accepted, although extra effort was needed to introduce the new instruments into clinical practice. Clinicians valued the clear structure of the guideline and found it useful for prognosis and treatment planning. Thus, the guideline will enable a more knowledge-based, effective and standardized stroke rehabilitation. This paper can be used as guidance by other stroke rehabilitation organizations, both national and international, when implementing evidence-based assessment in clinical praxis.

The authors thank all clinical reference group members: Susanne Alm (OT), Berit Bjerlemo (PT), Eva-Lena Bustrén (PT), Lena Christensson (OT), Ellen Hansson (OT), Rama Kiani (OT), Kirsten Lehmann (PT), Anton Mangsbo (PT), Malin Reinholdsson (PT), Pia Rydén (OT) and Elin Schlüter (PT). We also thank Hanna C Persson (PhD, PT) from the R&D unit and Gunilla Kjellby Wendt (PhD, PT) and Helga Hjort Buchholz (OT) from the management group for valuable input and support.

Funding. This work was partly supported by the Department of Occupational Therapy and Physiotherapy, Sahlgrenska University Hospital. No funding body had influence on the content of the work.

The authors have no conflicts of interest to declare.